Screen capture data amalgamation

ABSTRACT

At a device of a sender, a screen capture image is captured from a first portion of data presented on the device by a data source. Historical messaging data is analyzed to compute a set of topics in which the sender has participated. Using a topic from the set of topics as a context, a second portion of the data presented on the device by the data source is searched to extract an additional information from the second portion, the additional information corresponding to the context. the additional information is associated with the screen capture image. The screen capture image and the additional information are caused to be transmitted in a messaging channel.

TECHNICAL FIELD

The present invention relates generally to a method, system, andcomputer program product for capturing and sharing images from a screen.More particularly, the present invention relates to a method, system,and computer program product for screen capture data amalgamation.

BACKGROUND

A variety of data is presented on the display screens of a variety ofdevices. Technology exists to capture all or a portion of thepresentation on the display screen in the form of an image. An imageformed by capturing all or a portion of the data presented on a displayscreen is hereinafter referred to as a “screen capture image”, “screencapture”, or simply “image”, unless expressly distinguished where used.

A screen capture image is essentially a picture taken of all or aportion of the screen. This picture can include anything or any data ofany kind that is visibly presented on the screen at the time the screencapture image is captured or taken. For example, if a graphicalrendering, a picture or imagery, textual data, interactive data, a sceneof a video, a simulation, or some combination thereof is being visiblydisplayed on a portion of a screen at the instant of the capture, thescreen capture image will include an image of such a display on such aportion at such an instant.

The graphical rendering, the picture or imagery, the textual data, theinteractive data, the scene of a video, the simulation, or somecombination thereof is collectively and interchangeably referred toherein as “presented data”, “displayed data”, or simply “data”, unlessexpressly distinguished where used. Typically, a data source providesthe presented data that is being visibly displayed on a portion of ascreen at a given instant of the screen capture. As some non-limitingexamples, that data source may be a Uniform Resource Locator (URL), arepository, a webserver, an application server, and the like.

SUMMARY

The illustrative embodiments provide a method, system, and computerprogram product. An embodiment includes a method that captures a screencapture image at a device of a sender, from a first portion of datapresented on the device by a data source. The embodiment analyzeshistorical messaging data to compute a set of topics in which the senderhas participated. The embodiment searches, using a topic from the set oftopics as a context, a second portion of the data presented on thedevice by the data source. The embodiment extracts, responsive to thesearching, an additional information from the second portion, theadditional information corresponding to the context. The embodimentassociates the additional information with the screen capture image. Theembodiment causes the screen capture image and the additionalinformation to be transmitted in a messaging channel.

An embodiment includes a computer usable program product. The computerusable program product includes one or more computer-readable storagedevices, and program instructions stored on at least one of the one ormore storage devices.

An embodiment includes a computer system. The computer system includesone or more processors, one or more computer-readable memories, and oneor more computer-readable storage devices, and program instructionsstored on at least one of the one or more storage devices for executionby at least one of the one or more processors via at least one of theone or more memories.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features believed characteristic of the invention are setforth in the appended claims. The invention itself, however, as well asa preferred mode of use, further objectives and advantages thereof, willbest be understood by reference to the following detailed description ofthe illustrative embodiments when read in conjunction with theaccompanying drawings, wherein:

FIG. 1 depicts a block diagram of a network of data processing systemsin which illustrative embodiments may be implemented;

FIG. 2 depicts a block diagram of a data processing system in whichillustrative embodiments may be implemented;

FIG. 3 depicts a block diagram of an example configuration for screencapture data amalgamation in accordance with an illustrative embodiment;

FIG. 4 depicts a block diagram of another example configuration forscreen capture data amalgamation in accordance with an illustrativeembodiment;

FIG. 5 depicts a flowchart of an example process for screen capture dataamalgamation in accordance with an illustrative embodiment; and

FIG. 6 depicts a flowchart of another example process for screen capturedata amalgamation in accordance with an illustrative embodiment.

DETAILED DESCRIPTION

The illustrative embodiments recognize that screen capture imagesprovide a quick and easy way to share information with other users, butoften do not provide the additional detail or information that areceiver of the screen capture might need in relation to the sharedscreen capture image. A typical use-case of messaging between usersusing screen capture images occurs as follows—one user captures a screencapture image and sends to a receiving user with or without anyaccompanying sender-supplied annotations. The receiving user oftenresponds by asking the sender user one or more questions related to theimage. The sender user then searches for the answers to the askedquestion and messages the receiver with the answer related to the image.

For example, Jane Doe might find a table lamp online on a retailer'swebsite (a data source). The seller of the table lamp sends somepresented data to Jane Doe's computer or device, the presented dataincludes the image and additional information about the lamp, such asthe price, the size, the color, the materials used, any userestrictions, recommended wattage of bulbs, and so on. Jane Doe capturesa portion of the screen, e.g., the portion where the lamp's picture ispresented. Jane Doe sends to the screen captured image to John Doe via asuitable messaging channel.

A messaging channel is a method, application, system, or platform forelectronic communication. Instant messaging apps or platforms, socialmedia applications, and email are some non-limiting examples of amessaging channel within the scope of the illustrative embodiments.

In one use-case, John Doe receives the screen capture image and messagesback to Jane Doe asking for the material used in making the lamp in thepicture. Jane Doe revisits the retailer's website, finds (or does notfind) an answer to John Doe's question, and responds accordingly to JohnDoe.

In another use-case, suppose that the screen capture image shows notonly a lamp but also a picture frame placed adjacent to the lamp. WhileJane Doe intends to share the picture for the lamp, John Doe isinterested in the picture frame, and asks about the price of the pictureframe. Jane Doe revisits the retailer's website, finds (or does notfind) an answer to John Doe's question, and responds accordingly to JohnDoe.

It is possible that Jane and John Doe may have been communicatingpreviously about decorating a living area. Therefore, it is possiblethat the lamp and the picture frame, and the questions related to thosearticles, are in connection with the context of decorating the livingarea. As can be seen, simply sharing a screen capture image is ofteninsufficient, and requires further efforts on the part of the sender andthe receiver to communicate additional information relating to the imageor the context in which the image was captured and shared.

The illustrative embodiments recognize that the presently availabletools or solutions do not address these needs or provide adequatesolutions for these needs. The illustrative embodiments used to describethe invention generally address and solve the above-described problemsand other problems related to screen capture data amalgamation.

An embodiment can be implemented as a software application. Theapplication implementing an embodiment can be configured as amodification of an existing messaging channel or an imaging system, as aseparate application that operates in conjunction with an existingmessaging channel or an imaging system, a standalone application, orsome combination thereof.

An embodiment takes or receives a screen capture image of all or aportion of data source's data that is presented on a screen of thedevice where the embodiment is executing. An embodiment identifies acontext in which the screen capture image has been taken.

The identification of the context can take any one or more of thefollowing forms—In one case, the sender user, i.e., the user whocaptures the screen capture image, may have certain interests. Forexample, the sender user may be Jane Doe may be an interior decorator,who may be interested in interior decoration, lighting, and otherappliances and fixtures. As another example, another user may be aphotographer who may be interested in photographs, printed art, framedpictures, and the like. Accordingly, one embodiment analyzes the dataavailable about the sender user, e.g., from past communications of thesender with one or more other users, to determine the sender user'sinterests. Any one or more of the determined interests is usable as acontext.

In another case, the sender may have been previously engaged in aconversation with another user over a messaging channel. As an example,a topic of discussion in the historical messages between the sender andthe other user may be interior decoration of a living space. The sendermay be engaged in another conversation with a different user about, say,aviation-related topics. Any one or more of the determined topics isusable as a context. When an embodiment analyzes a screen capture imageto determine objects or themes that might be present in the screencapture image, the embodiment correlates the determined objects orthemes with a context of a communication. An embodiment uses the topicthat relates to the object or theme of the screen capture image as acontext. The embodiment uses the context to also identify a likelyreceiver of the screen capture image.

In another case, the sender may have already selected a receiving userfor the screen capture image. another embodiment analyzes a previousconversation with the identified receiver user over a messaging channel.As an example, a topic of discussion in the historical messages betweenthe sender and the receiver user may be interior decoration of a livingspace. One or more of the determined topics is usable as a context. Whenan embodiment analyzes a screen capture image to determine objects orthemes that might be present in the screen capture image, the embodimentcorrelates the determined objects or themes with a context of acommunication. An embodiment uses the topic that relates to the objector theme of the screen capture image as a context.

Any number of contexts may apply to a screen capture image. Any numberof messaging channels may provide the historical messaging data. Anynumber of likely recipients of the screen capture image may beidentified.

An embodiment further associates a confidence value or confidence weightwith a context determination, a likely receiver determination, or both.Any known method of cognitive analysis of the screen capture image andthe historical messaging data can be used to compute the confidencevalue. The data of the actual messaging that occurs between the senderand a receiver subsequent to sending the screen capture image can beused as feedback in any known machine-learning methodology to train orimprove the training of said cognitive analysis method.

Prior to sending the screen capture image, and after a context has beenestablished for a screen capture image, an embodiment analyzes thepresented data of the data source from which the screen capture imagewas created. Particularly, the embodiment analyzes the presented data toidentify additional information in other portions of the presenteddata—which are not included in the screen capture image—the informationbeing related to the context.

For instance, in the example context of decorating a living space,things have to be bought to decorate the living space. Accordingly, aprice of an object that is usable in the decorating and that is presentin the screen capture image is an additional information that is relatedto the context. The presented data may have a textual description of theexample lamp described herein, and a pricing information may be found inthe textual description.

Operating in this manner, the embodiment identifies some additionalinformation that is related to the context. The embodiment populates themetadata of the screen capture image with the identified additionalinformation.

An embodiment sends the metadata with the screen capture image, orcauses the sending of the metadata with the screen capture image, whenthe screen capture image is actually sent to a receiver over a messagingchannel. For example, if the messaging channel is an instant messagingapp, the embodiment causes the metadata to be communicated to thereceiver together with, or soon after, the message containing the screencapture image. Note that the determination of the additionalinformation, associating the additional information with the screencapture image, and the sending of the additional information with thescreen capture image are all performed without the intervention oraction on the part of the sender.

Subsequent to the sending of the screen capture image and the metadata,the receiver may still have questions or may otherwise express a needfor further information about the screen capture image. An embodimentanalyzes such subsequent messaging between the receiver and the sender.The embodiment identifies a new topic or context of the subsequentmessaging. In a manner described herein, the embodiment further analyzesthe presented data from which the screen capture image was created. Thisfurther analysis is performed to identify additional information thatrelates to the new context.

An embodiment associates the additional information relating to the newcontext as additional metadata with the screen capture image. Theembodiment sends or causes the sending of the additional metadataautomatically in the subsequent messaging in a manner described herein.

The manner of screen capture data amalgamation described herein isunavailable in the presently available methods. A method of anembodiment described herein, when implemented to execute on a device ordata processing system, comprises substantial advancement of thefunctionality of that device or data processing system in automaticallydetermining a need for additional contextual information related to ascreen capture image, and automatically supplying such additionalinformation in a communication where the screen capture image is sent.The determining and sending can be ongoing during a conversation relatedto the screen capture image.

The illustrative embodiments are described with respect to certain typesof data source, presented data and portions thereof, screen captureimage, topics, objects, contexts, analyses, values and weights,messages, messaging channels, additional information, metadata, devices,data processing systems, environments, components, and applications onlyas examples. Any specific manifestations of these and other similarartifacts are not intended to be limiting to the invention. Any suitablemanifestation of these and other similar artifacts can be selectedwithin the scope of the illustrative embodiments.

Furthermore, the illustrative embodiments may be implemented withrespect to any type of data, data source, or access to a data sourceover a data network. Any type of data storage device may provide thedata to an embodiment of the invention, either locally at a dataprocessing system or over a data network, within the scope of theinvention. Where an embodiment is described using a mobile device orwireless device, any type of data storage device suitable for use withthe mobile device may provide the data to such embodiment, eitherlocally at the mobile device or over a data network, within the scope ofthe illustrative embodiments.

The illustrative embodiments are described using specific code, designs,architectures, protocols, layouts, schematics, and tools only asexamples and are not limiting to the illustrative embodiments.Furthermore, the illustrative embodiments are described in someinstances using particular software, tools, and data processingenvironments only as an example for the clarity of the description. Theillustrative embodiments may be used in conjunction with othercomparable or similarly purposed structures, systems, applications, orarchitectures. For example, other comparable mobile devices, structures,systems, applications, or architectures therefor, may be used inconjunction with such embodiment of the invention within the scope ofthe invention. An illustrative embodiment may be implemented inhardware, software, or a combination thereof.

The examples in this disclosure are used only for the clarity of thedescription and are not limiting to the illustrative embodiments.Additional data, operations, actions, tasks, activities, andmanipulations will be conceivable from this disclosure and the same arecontemplated within the scope of the illustrative embodiments.

Any advantages listed herein are only examples and are not intended tobe limiting to the illustrative embodiments. Additional or differentadvantages may be realized by specific illustrative embodiments.Furthermore, a particular illustrative embodiment may have some, all, ornone of the advantages listed above.

With reference to the figures and in particular with reference to FIGS.1 and 2, these figures are example diagrams of data processingenvironments in which illustrative embodiments may be implemented. FIGS.1 and 2 are only examples and are not intended to assert or imply anylimitation with regard to the environments in which differentembodiments may be implemented. A particular implementation may makemany modifications to the depicted environments based on the followingdescription.

FIG. 1 depicts a block diagram of a network of data processing systemsin which illustrative embodiments may be implemented. Data processingenvironment 100 is a network of computers in which the illustrativeembodiments may be implemented. Data processing environment 100 includesnetwork 102. Network 102 is the medium used to provide communicationslinks between various devices and computers connected together withindata processing environment 100. Network 102 may include connections,such as wire, wireless communication links, or fiber optic cables.

Clients or servers are only example roles of certain data processingsystems connected to network 102 and are not intended to exclude otherconfigurations or roles for these data processing systems. Server 104and server 106 couple to network 102 along with storage unit 108.Software applications may execute on any computer in data processingenvironment 100. Clients 110, 112, and 114 are also coupled to network102. A data processing system, such as server 104 or 106, or client 110,112, or 114 may contain data and may have software applications orsoftware tools executing thereon.

Only as an example, and without implying any limitation to sucharchitecture, FIG. 1 depicts certain components that are usable in anexample implementation of an embodiment. For example, servers 104 and106, and clients 110, 112, 114, are depicted as servers and clients onlyas example and not to imply a limitation to a client-serverarchitecture. As another example, an embodiment can be distributedacross several data processing systems and a data network as shown,whereas another embodiment can be implemented on a single dataprocessing system within the scope of the illustrative embodiments. Dataprocessing systems 104, 106, 110, 112, and 114 also represent examplenodes in a cluster, partitions, and other configurations suitable forimplementing an embodiment.

Device 132 is an example of a device described herein. For example,device 132 can take the form of a smartphone, a tablet computer, alaptop computer, client 110 in a stationary or a portable form, awearable computing device, or any other suitable device. Any softwareapplication described as executing in another data processing system inFIG. 1 can be configured to execute in device 132 in a similar manner.Any data or information stored or produced in another data processingsystem in FIG. 1 can be configured to be stored or produced in device132 in a similar manner.

Client 114 is usable by a sender user to send a screen capture image ina manner described herein. Application 115 implements an embodimentdescribed herein. Application 115 interacts with messaging application116 to perform one or more operations described herein. Data source 107provides presented data on client 114, using which the screen captureimage is created on client 114, and using which application 115constructs the metadata of the screen capture image as described herein.The embodiments can be implemented on any type of device usable forcommunicating using a messaging channel. For example, device 132 can beconfigured in a manner similar to client 114, where application 134 andmessaging application 136 perform functions similar to application 115and messaging application 116, respectively. Data source 107 presentsdata on device 132 in a manner similar to the presented data on client114.

Servers 104 and 106, storage unit 108, and clients 110, 112, and 114,and device 132 may couple to network 102 using wired connections,wireless communication protocols, or other suitable data connectivity.Clients 110, 112, and 114 may be, for example, personal computers ornetwork computers.

In the depicted example, server 104 may provide data, such as bootfiles, operating system images, and applications to clients 110, 112,and 114. Clients 110, 112, and 114 may be clients to server 104 in thisexample. Clients 110, 112, 114, or some combination thereof, may includetheir own data, boot files, operating system images, and applications.Data processing environment 100 may include additional servers, clients,and other devices that are not shown.

In the depicted example, data processing environment 100 may be theInternet. Network 102 may represent a collection of networks andgateways that use the Transmission Control Protocol/Internet Protocol(TCP/IP) and other protocols to communicate with one another. At theheart of the Internet is a backbone of data communication links betweenmajor nodes or host computers, including thousands of commercial,governmental, educational, and other computer systems that route dataand messages. Of course, data processing environment 100 also may beimplemented as a number of different types of networks, such as forexample, an intranet, a local area network (LAN), or a wide area network(WAN). FIG. 1 is intended as an example, and not as an architecturallimitation for the different illustrative embodiments.

Among other uses, data processing environment 100 may be used forimplementing a client-server environment in which the illustrativeembodiments may be implemented. A client-server environment enablessoftware applications and data to be distributed across a network suchthat an application functions by using the interactivity between aclient data processing system and a server data processing system. Dataprocessing environment 100 may also employ a service orientedarchitecture where interoperable software components distributed acrossa network may be packaged together as coherent business applications.Data processing environment 100 may also take the form of a cloud, andemploy a cloud computing model of service delivery for enablingconvenient, on-demand network access to a shared pool of configurablecomputing resources (e.g. networks, network bandwidth, servers,processing, memory, storage, applications, virtual machines, andservices) that can be rapidly provisioned and released with minimalmanagement effort or interaction with a provider of the service.

With reference to FIG. 2, this figure depicts a block diagram of a dataprocessing system in which illustrative embodiments may be implemented.Data processing system 200 is an example of a computer, such as servers104 and 106, or clients 110, 112, and 114 in FIG. 1, or another type ofdevice in which computer usable program code or instructionsimplementing the processes may be located for the illustrativeembodiments.

Data processing system 200 is also representative of a data processingsystem or a configuration therein, such as data processing system 132 inFIG. 1 in which computer usable program code or instructionsimplementing the processes of the illustrative embodiments may belocated. Data processing system 200 is described as a computer only asan example, without being limited thereto. Implementations in the formof other devices, such as device 132 in FIG. 1, may modify dataprocessing system 200, such as by adding a touch interface, and eveneliminate certain depicted components from data processing system 200without departing from the general description of the operations andfunctions of data processing system 200 described herein.

In the depicted example, data processing system 200 employs a hubarchitecture including North Bridge and memory controller hub (NB/MCH)202 and South Bridge and input/output (I/O) controller hub (SB/ICH) 204.Processing unit 206, main memory 208, and graphics processor 210 arecoupled to North Bridge and memory controller hub (NB/MCH) 202.Processing unit 206 may contain one or more processors and may beimplemented using one or more heterogeneous processor systems.Processing unit 206 may be a multi-core processor. Graphics processor210 may be coupled to NB/MCH 202 through an accelerated graphics port(AGP) in certain implementations.

In the depicted example, local area network (LAN) adapter 212 is coupledto South Bridge and I/O controller hub (SB/ICH) 204. Audio adapter 216,keyboard and mouse adapter 220, modem 222, read only memory (ROM) 224,universal serial bus (USB) and other ports 232, and PCI/PCIe devices 234are coupled to South Bridge and I/O controller hub 204 through bus 238.Hard disk drive (HDD) or solid-state drive (SSD) 226 and CD-ROM 230 arecoupled to South Bridge and I/O controller hub 204 through bus 240.PCI/PCIe devices 234 may include, for example, Ethernet adapters, add-incards, and PC cards for notebook computers. PCI uses a card buscontroller, while PCIe does not. ROM 224 may be, for example, a flashbinary input/output system (BIOS). Hard disk drive 226 and CD-ROM 230may use, for example, an integrated drive electronics (IDE), serialadvanced technology attachment (SATA) interface, or variants such asexternal-SATA (eSATA) and micro-SATA (mSATA). A super I/O (SIO) device236 may be coupled to South Bridge and I/O controller hub (SB/ICH) 204through bus 238.

Memories, such as main memory 208, ROM 224, or flash memory (not shown),are some examples of computer usable storage devices. Hard disk drive orsolid state drive 226, CD-ROM 230, and other similarly usable devicesare some examples of computer usable storage devices including acomputer usable storage medium.

An operating system runs on processing unit 206. The operating systemcoordinates and provides control of various components within dataprocessing system 200 in FIG. 2. The operating system may be acommercially available operating system for any type of computingplatform, including but not limited to server systems, personalcomputers, and mobile devices. An object oriented or other type ofprogramming system may operate in conjunction with the operating systemand provide calls to the operating system from programs or applicationsexecuting on data processing system 200.

Instructions for the operating system, the object-oriented programmingsystem, and applications or programs, such as application 115 orapplication 134 in FIG. 1, are located on storage devices, such as inthe form of code 226A on hard disk drive 226, and may be loaded into atleast one of one or more memories, such as main memory 208, forexecution by processing unit 206. The processes of the illustrativeembodiments may be performed by processing unit 206 using computerimplemented instructions, which may be located in a memory, such as, forexample, main memory 208, read only memory 224, or in one or moreperipheral devices.

Furthermore, in one case, code 226A may be downloaded over network 201Afrom remote system 201B, where similar code 201C is stored on a storagedevice 201D. in another case, code 226A may be downloaded over network201A to remote system 201B, where downloaded code 201C is stored on astorage device 201D.

The hardware in FIGS. 1-2 may vary depending on the implementation.Other internal hardware or peripheral devices, such as flash memory,equivalent non-volatile memory, or optical disk drives and the like, maybe used in addition to or in place of the hardware depicted in FIGS.1-2. In addition, the processes of the illustrative embodiments may beapplied to a multiprocessor data processing system.

In some illustrative examples, data processing system 200 may be apersonal digital assistant (PDA), which is generally configured withflash memory to provide non-volatile memory for storing operating systemfiles and/or user-generated data. A bus system may comprise one or morebuses, such as a system bus, an I/O bus, and a PCI bus. Of course, thebus system may be implemented using any type of communications fabric orarchitecture that provides for a transfer of data between differentcomponents or devices attached to the fabric or architecture.

A communications unit may include one or more devices used to transmitand receive data, such as a modem or a network adapter. A memory may be,for example, main memory 208 or a cache, such as the cache found inNorth Bridge and memory controller hub 202. A processing unit mayinclude one or more processors or CPUs.

The depicted examples in FIGS. 1-2 and above-described examples are notmeant to imply architectural limitations. For example, data processingsystem 200 also may be a tablet computer, laptop computer, or telephonedevice in addition to taking the form of a mobile or wearable device.

Where a computer or data processing system is described as a virtualmachine, a virtual device, or a virtual component, the virtual machine,virtual device, or the virtual component operates in the manner of dataprocessing system 200 using virtualized manifestation of some or allcomponents depicted in data processing system 200. For example, in avirtual machine, virtual device, or virtual component, processing unit206 is manifested as a virtualized instance of all or some number ofhardware processing units 206 available in a host data processingsystem, main memory 208 is manifested as a virtualized instance of allor some portion of main memory 208 that may be available in the hostdata processing system, and disk 226 is manifested as a virtualizedinstance of all or some portion of disk 226 that may be available in thehost data processing system. The host data processing system in suchcases is represented by data processing system 200.

With reference to FIG. 3, this figure depicts a block diagram of anexample configuration for screen capture data amalgamation in accordancewith an illustrative embodiment. Application 302 is an example ofapplication 115 or 134 in FIG. 1, and accordingly, data processingsystem 304 corresponds to client 114 or device 132, respectively, inFIG. 1. Data source 306 is an example of data source 107 in FIG. 1.

Only as a non-limiting example for the clarity of the description,suppose that data source 306 presents data using URL 308 in browser 310.The body of browser 310 contains presented data 312. Further supposethat a screen capture operation captures portion 314 of presented data312 as a screen capture image. The remaining portion of presented data312—which is the portion of presented data 312 other than the portioncaptured in the screen capture image—may fully or partially be visiblein the body of browser 310. For example, some of the remaining portionmay be reachable in the body of browser 310 by scrolling. Information316A (price of an article depicted in presented data 312), information316B (size of the article depicted in presented data 312), andinformation (construction material of the article depicted in presenteddata 312) are some examples of information available in the remainingportion of presented data 312.

In any of the manners described herein, application 302 determines acontext associated with the screen capture image captured from portion314. As an example, application 302 analyzes the remaining portion anddetermines that information 316A and 316B are related to the context.Accordingly, application 302 populates screen capture image 318 withinformation 316A and 316B. Application 302 then causes the metadata tobe sent in conjunction with screen capture image 318 in the form ofmessage 320.

Thus, as a part of communication 1 (labelled with circled 1),application 302 causes screen capture image 318 and metadata 320 to besent from messaging channel 321 operating in data processing system 304of the sender user, to messaging channel 322 operating in dataprocessing system 324 of a receiver user. Metadata 320 automaticallyprovided in communication 1 may satisfy the receiver user's need foradditional information related to screen capture image 318.

In one example case described herein, the receiver user may need moreinformation about screen capture image 318. For example, the receiveruser may send communication 2 (labelled with circled 2) to the senderuser's system 304. Communication 2 includes message 326 (shown only inthe receiver-side messaging channel 322 and not in sender-side messagingchannel 321 for clarity).

As a result of receiving communication 2 at data processing system 304,application 302 performs an analysis of message 326 to compute a new oradditional context. In the depicted example, suppose that the additionalcontext is “material”. Using the additional context, application 302extracts additional information 316C from the remaining portion ofpresented data 312. Application 302 causes additional information 316Cto be associated as additional metadata with screen capture image 318.Application 302 causes the additional metadata to be sent ascommunication 3 (labelled with circled 3). In the depicted non-limitingexample, communication 3 appears as additional message 328 in messagingchannel 322.

Note that the browser-method of presenting data, the scrolling method ofreaching various parts of the remaining portion, presenting the metadataas one or more separate messages, and other depicted operations are usedonly as non-limiting example ways of accomplishing certain operationsdescribed herein. From this disclosure, those of ordinary skill in theart will be able to conceive many other ways of accomplishing theseoperations and the same are contemplated within the scope of theillustrative embodiments. For example, presented data 312 may be loadedin a document editor instead of a browser, the remaining portion may bepaginated instead of scrollable, and the metadata may be shown andpresented in a messaging channel in or with the screen capture imageinstead of a separate message.

With reference to FIG. 4, this figure depicts a block diagram of anotherexample configuration for screen capture data amalgamation in accordancewith an illustrative embodiment. Application 402 is an example ofapplication 302 in FIG. 3. Messaging data 404 is data obtained from themessaging channel, such as messaging channel 321 in FIG. 3, used by thesender user for communicating with one or more other users. Note thatmore than one messaging channels may provide messaging data 404.Presented data 406 is an example of presented data 312 in FIG. 3.

A screen capture image may be created by component 408 or anothercomponent operating outside application 402. Component 410 analyzeshistorical messaging data to extract one or more contexts as describedherein. For example, an extracted context may be a topic that the senderis expected to be interested in, with a certain confidence value. Asanother example, an extracted context may be a topic that the sender andan expected receiver are likely to be interested in, with a certainconfidence value. As another example, an extracted context may be atopic that the sender and an identified receiver are determined to havebeen interested in, with a certain confidence value.

Component 412 parses, analyzes, or otherwise performs machineunderstanding of the remaining portion of presented data 406, i.e. theportion other than the portion included in the screen capture image.Using this machine understanding of the remaining portion, component 412identifies in presented data 406 one or more pieces of additionalinformation related to the screen capture image.

Component 414 suitably associates the identified pieces of additionalinformation with the screen capture image. One non-limiting manner ofassociating the identified pieces of additional information with thescreen capture image includes making the additional information a partof the metadata of the screen capture image. The association between theadditional information and the screen capture image can be formed inways other than the image metadata, such as by embedding the informationin the image graphic, or including the information in a file that isseparate from the file containing the image data.

Component 420 causes the additional information to be sent inconjunction with the screen capture image in a messaging channel. Ifsubsequent messaging occurs in the messaging channel, component 418analyses the subsequent messaging data to determine additional contextas described herein. If more or different additional information isneeded according to the additional context, component 416 uses component412 to extract such more or different pieces of additional information.Component 418 operates in the manner of component 414 to associate themore or different pieces of additional information with the screencapture image. Component 420 communicates the more or different piecesof additional information in the messaging channel.

With reference to FIG. 5, this figure depicts a flowchart of an exampleprocess for screen capture data amalgamation in accordance with anillustrative embodiment. Process 500 can be implemented in application402 in FIG. 4.

The application takes or receives a screen capture image from a datapresentation of a data source, e.g., presented data 406 in FIG. 4 (block502). The application identifies a likely or actual recipient of thescreen capture image as the case may be, in a manner described herein(block 504).

The application analyzes historical messaging data from a past period inone or more messaging channels to determine a set of contexts and acorresponding set of their confidence values as described herein (block506). The application selects a historical context computed in theanalysis of block 506 (block 508). For example, the application mayselect that historical context which has the highest confidence value,or the application may select a context whose confidence value exceeds athreshold confidence value. Other methods for selecting a context basedon the confidence values will be apparent from this disclosure to thoseof ordinary skill in the art and the same are contemplated within thescope of the illustrative embodiments.

The application analyzes the remaining portion of the presented data ina manner described herein (block 510). From the analysis of block 510,the application selects one or more pieces of additional data thatcorrespond to the selected context (block 512). The application adds orpopulates the additional information as metadata of the screen captureimage, or associates the additional information with the screen captureimage in other suitable manner (block 514).

The application sends or causes the sending of the additionalinformation with the screen capture image to a receiver (block 516). Theapplication either ends process 500 thereafter or exits process 500 viaexit point label “A” to enter process 600 via entry point labelled “A”therein.

With reference to FIG. 6, this figure depicts a flowchart of anotherexample process for screen capture data amalgamation in accordance withan illustrative embodiment. Process 600 can be implemented inapplication 402 in FIG. 4. The application optionally enters process 600from process 500 of FIG. 5 via entry point A.

The application receives or detects receiving subsequent message dataafter sending the screen capture image and additional information fromprocess 500 (block 602). The application analyzes the subsequent messagedata to determine an additional context as described herein (block 604).Optionally, as a result of identifying additional context in theanalysis of block 604, the application may adjust or modify a confidenceweighting algorithm in a cognitive analysis system as described herein(block 606).

The application analyzes the remaining portion of the presented data tofind further additional information that is related to the additionalcontext (block 608). The application selects the further additionalinformation corresponding to the context, if such information is foundin the presented data (block 610).

The application adds or populates the further additional information asadditional metadata of the screen capture image, or associates thefurther additional information with the screen capture image in othersuitable manner (block 612).

The application sends or causes the sending of the further additionalinformation in relation to the screen capture image in the messagingchannel (block 614). If no further additional information is found, theapplication may optionally cause the sending of a negative resultmessage advising the receiver that no additional information relating tothe additional context was found in the presented data. The applicationends process 600 thereafter.

Thus, a computer implemented method, system or apparatus, and computerprogram product are provided in the illustrative embodiments for screencapture data amalgamation and other related features, functions, oroperations. Where an embodiment or a portion thereof is described withrespect to a type of device, the computer implemented method, system orapparatus, the computer program product, or a portion thereof, areadapted or configured for use with a suitable and comparablemanifestation of that type of device.

Where an embodiment is described as implemented in an application, thedelivery of the application in a Software as a Service (SaaS) model iscontemplated within the scope of the illustrative embodiments. In a SaaSmodel, the capability of the application implementing an embodiment isprovided to a user by executing the application in a cloudinfrastructure. The user can access the application using a variety ofclient devices through a thin client interface such as a web browser(e.g., web-based e-mail), or other light-weight client-applications. Theuser does not manage or control the underlying cloud infrastructureincluding the network, servers, operating systems, or the storage of thecloud infrastructure. In some cases, the user may not even manage orcontrol the capabilities of the SaaS application. In some other cases,the SaaS implementation of the application may permit a possibleexception of limited user-specific application configuration settings.

The present invention may be a system, a method, and/or a computerprogram product at any possible technical detail level of integration.The computer program product may include a computer readable storagemedium (or media) having computer readable program instructions thereonfor causing a processor to carry out aspects of the present invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, configuration data for integrated circuitry, oreither source code or object code written in any combination of one ormore programming languages, including an object oriented programminglanguage such as Smalltalk, C++, or the like, and procedural programminglanguages, such as the “C” programming language or similar programminglanguages. The computer readable program instructions may executeentirely on the user's computer, partly on the user's computer, as astand-alone software package, partly on the user's computer and partlyon a remote computer or entirely on the remote computer or server. Inthe latter scenario, the remote computer may be connected to the user'scomputer through any type of network, including a local area network(LAN) or a wide area network (WAN), or the connection may be made to anexternal computer (for example, through the Internet using an InternetService Provider). In some embodiments, electronic circuitry including,for example, programmable logic circuitry, field-programmable gatearrays (FPGA), or programmable logic arrays (PLA) may execute thecomputer readable program instructions by utilizing state information ofthe computer readable program instructions to personalize the electroniccircuitry, in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the blocks may occur out of theorder noted in the Figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

What is claimed is:
 1. A method comprising: capturing, at a device of asender, from a first portion of data presented on the device by a datasource, a screen capture image, the data being presented duringcommunication on a messaging channel, the messaging channel comprising asystem for electronic communication; extracting a context and aconfidence value of the context by analyzing historical topics ofcommunications of the sender, the communications of the sender sent inthe messaging channel, the context comprising a topic selected from aset of topics within the communications of the sender, the confidencevalue computed using a cognitive analysis method; extracting additionalinformation related to the context from a second portion of the datapresented on the device by the data source, the second portioncomprising data not captured in the screen capture image; associatingthe additional information with the screen capture image, theassociating comprising modifying the screen capture image to embed theadditional information into the screen capture image; causing the screencapture image and the additional information to be transmitted in themessaging channel; detecting, subsequent to the causing, a secondcommunication in the messaging channel; and causing the screen captureimage, the additional information, and the second communication to beused to improve the cognitive analysis method.
 2. The method of claim 1,further comprising: detecting, responsive to the causing, a subsequentcommunication in the messaging channel; analyzing data of the subsequentcommunication to extract a second context; searching, using the secondcontext, the second portion to locate a second additional information,the second additional information corresponding to the second context;associating the second additional information with the screen captureimage; and causing the second additional information to be transmittedrelative to the screen capture image in the messaging channel.
 3. Themethod of claim 1, further comprising: detecting, responsive to thecausing, a subsequent communication in the messaging channel; analyzingdata of the subsequent communication to extract a second context;searching, using the second context, the second portion to locate asecond additional information, the second additional informationcorresponding to the second context; failing to locate the secondadditional information in the second portion; and causing a messageabout the failing to be transmitted relative to the screen capture imagein the messaging channel.
 4. The method of claim 1, further comprising:adding, as a part of the associating, the additional information asmetadata of the screen capture image.
 5. The method of claim 1, furthercomprising: determining from analyzing the historical topics ofcommunications of the sender, a likely receiver of the screen captureimage; and selecting the topic, responsive to the sender communicatingwith the likely receiver about the topic, from the historical topics ofcommunications of the sender.
 6. The method of claim 1, furthercomprising: identifying a specified receiver of the screen captureimage; and selecting the topic, responsive to the sender communicatingwith the specified receiver about the topic, from the historical topicsof communications of the sender.
 7. The method of claim 1, wherein thefirst portion of the data is visibly presented on a display screenassociated with the device, and wherein the second portion is availableat the device.
 8. The method of claim 7, wherein the second portion isnot visible on the device.
 9. The method of claim 1, further comprising:computing a past period, wherein the historical topics are present inhistorical messaging data of the past period.
 10. A computer usableprogram product comprising a computer readable storage medium, andprogram instructions stored on the computer readable storage medium, thestored program instructions comprising: program instructions to capture,at a device of a sender, from a first portion of data presented on thedevice by a data source, a screen capture image, the data beingpresented during communication on a messaging channel, the messagingchannel comprising a system for electronic communication; programinstructions to extract a context and a confidence value of the contextby analyzing historical topics of communications of the sender, thecommunications of the sender sent in the messaging channel, the contextcomprising a topic selected from a set of topics within thecommunications of the sender, the confidence value computed using acognitive analysis method; program instructions to extract additionalinformation related to the context from a second portion of the datapresented on the device by the data source, the second portioncomprising data not captured in the screen capture image; programinstructions to associate the additional information with the screencapture image, the associating comprising modifying the screen captureimage to embed the additional information into the screen capture image;program instructions to cause the screen capture image and theadditional information to be transmitted in the messaging channel;program instructions to detect, subsequent to the causing, a secondcommunication in the messaging channel; and program instructions tocause the screen capture image, the additional information, and thesecond communication to be used to improve the cognitive analysismethod.
 11. The computer usable program product of claim 10, furthercomprising: program instructions to detect, responsive to the causing, asubsequent communication in the messaging channel; program instructionsto analyze data of the subsequent communication to extract a secondcontext; program instructions to search, using the second context, thesecond portion to locate a second additional Previously Presentedinformation, the second additional Previously Presented informationcorresponding to the second context; program instructions to associatethe second additional information with the screen capture image; andprogram instructions to cause the second additional information to betransmitted relative to the screen capture image in the messagingchannel.
 12. The computer usable program product of claim 10, furthercomprising: program instructions to detect, responsive to the causing, asubsequent communication in the messaging channel; program instructionsto analyze data of the subsequent communication to extract a secondcontext; program instructions to search, using the second context, thesecond portion to locate a second additional information, the secondadditional information corresponding to the second context; programinstructions to fail to locate the second additional information in thesecond portion; and program instructions to cause a message about thefailing to be transmitted relative to the screen capture image in themessaging channel.
 13. The computer usable program product of claim 10,further comprising: program instructions to add, as a part of theassociating, the additional information as metadata of the screencapture image.
 14. The computer usable program product of claim 10,further comprising: program instructions to determine from analyzing thehistorical topics of communications of the sender, a likely receiver ofthe screen capture image; and program instructions to select the topic,responsive to the sender communicating with the likely receiver aboutthe topic, from the historical topics of communications of the sender.15. The computer usable program product of claim 10, further comprising:program instructions to identify a specified receiver of the screencapture image; and program instructions to select the topic, responsiveto the sender communicating with the specified receiver about the topic,from the historical topics of communications of the sender.
 16. Thecomputer usable program product of claim 10, wherein the computer usablecode is stored in a computer readable storage device in a dataprocessing system, and wherein the computer usable code is transferredover a network from a remote data processing system.
 17. The computerusable program product of claim 10, wherein the computer usable code isstored in a computer readable storage device in a server data processingsystem, and wherein the computer usable code is downloaded over anetwork to a remote data processing system for use in a computerreadable storage device associated with the remote data processingsystem.
 18. A computer system comprising a processor, acomputer-readable memory, and a computer readable storage medium, andprogram instructions stored on the computer readable storage medium forexecution by the processor via the memory, the stored programinstructions comprising: program instructions to capture, at a device ofa sender, from a first portion of data presented on the device by a datasource, a screen capture image, the data being presented duringcommunication on a messaging channel, the messaging channel comprising asystem for electronic communication; program instructions to extract acontext and a confidence value of the context by analyzing historicaltopics of communications of the sender, the communications of the sendersent in the messaging channel, the context comprising a topic selectedfrom a set of topics within the communications of the sender, theconfidence value computed using a cognitive analysis method; programinstructions to extract additional information related to the contextfrom a second portion of the data presented on the device by the datasource, the second portion comprising data not captured in the screencapture image; program instructions to associate the additionalinformation with the screen capture image, the associating comprisingmodifying the screen capture image to embed the additional informationinto the screen capture image; program instructions to cause the screencapture image and the additional information to be transmitted in themessaging channel; program instructions to detect, subsequent to thecausing, a second communication in the messaging channel; and programinstructions to cause the screen capture image, the additionalinformation, and the second communication to be used to improve thecognitive analysis method.